Performance and limitations of the linguistically motivated Cocoa/Peaberry system in a broad biological domain

We tested a linguistically motivated rulebased system in the Cancer Genetics task of the BioNLP13 shared task challenge. The performance of the system was very moderate, ranging from 52% against the development set to 45% against the test set. Interestingly, the performance of the system did not change appreciably when using only entities tagged by the inbuilt tagger as compared to performance using the gold-tagged entities. The lack of an event anaphoric module, as well as problems in reducing events generated by a large trigger class to the task-specific event subset, were likely major contributory factors to the rather moderate performance.